Probabilistic Modeling of Bowing Gestures for Gesture-based Violin Sound Synthesis

We present a probabilistic approach to modeling violin bowing gestures, for the purpose of synthesizing violin sound from a musical score. The gesture models are based on Gaussian processes, a principled probabilistic framework. Models for bow velocity, bow-bridge distance and bow force during a stroke are learned from training data of recorded bowing motion. From the models of bow motion during a stroke, slightly novel bow motion can be synthesized, varying in a random manner along the main modes of variation learned from the data. Such synthesized bow strokes can be stitched together to form a continuous bowing motion, which can drive a physical violin model, producing naturalistic violin sound. Listening tests show that the sound produced from the synthetic bowing motion is perceived as very similar to sound produced from real bowing motion, recorded with motion capture. Even more importantly, the Gaussian process framework allows modeling short and long range temporal dependencies, as well as learning latent style parameters from the training data in an unsupervised manner.